Electronic device, wearable device, and method for controlling object displayed through electronic device

ABSTRACT

Disclosed are an electronic device and method for controlling objects displayed through the electronic device. A method includes displaying a stereoscopic screen including a left-eye screen and a right-eye screen through a display of the electronic device, moving a pointer object in the stereoscopic screen, based on a movement of the electronic device sensed by a sensor of the electronic device, identifying whether the pointer object is positioned at a first position for a predetermined time in the stereoscopic screen, based on information obtained from the sensor, and based on identifying that the pointer object is positioned at the first position for the predetermined time, moving at least one object included in the stereoscopic screen into the first position at which the pointer object is positioned.

PRIORITY

This application is a continuation of U.S. patent application Ser. No.17/088,106, which was filed in the U.S. Patent and Trademark Office(USPTO) on Nov. 3, 2020, which is a continuation of U.S. patentapplication Ser. No. 15/343,732, which was filed in the USPTO on Nov. 4,2016, issued as U.S. Pat. No. 10,824,236 on Nov. 3, 2020, and claimspriority under 35 U.S.C. § 119(a) to Korean Patent Application SerialNo. 10-2015-0154732, which was filed in the Korean Intellectual PropertyOffice on Nov. 4, 2015, the entire content of each of which isincorporated herein by reference.

BACKGROUND 1. Field of the Disclosure

The present disclosure relates generally to electronic devices, and moreparticularly, to wearable electronic devices, and methods forcontrolling objects displayed through such electronic devices.

2. Description of the Related Art

Many electronic devices that are wearable on the human body have beenintroduced. Such electronic devices are typically referred to aswearable devices and are provided in various types, such as headmounted, glasses, watches, bands, contact lenses, rings, shoes, clothes,or other manners to be worn on the human body or clothes, leading toincreased portability and accessibility.

Head mounted wearable devices, such as head mounted displays (HMD) arebeing rapidly developed. An HMD may provide images in a see-through typeproviding augmented reality (AR) and in a see-closed type providingvirtual reality (VR).

The see-through type may synthesize or combine a virtual object ortarget with a real-life base using the characteristics of asemi-transmissive lens to reinforce additional information that may bedifficult to obtain otherwise. The see-closed type places two displaysahead of the user's eyes and allows the user alone to view contents,such as games, movies, streaming, or broadcast provided through anexternal input through an independent screen, so that the user may focuson only the contents.

Under research as per existing technology are methods allowing anelectronic device such as a smartphone equipped with a separate displayto be mounted on a wearable device as a display means for the wearabledevice.

Upon mounting an electronic device on a wearable device for use, datarelated to a user's entry, such as a motion acceleration, may beconsidered to control objects outputted through the display of theelectronic device. To this point, however, the prior art is deficient inthe development of wearable device technology that considers such data,giving rise to a need in the art for improvement in the control ofobjects outputted through the electronic device display.

SUMMARY

Aspects of the present disclosure are to address at least the abovementioned problems and/or disadvantages and to provide at least theadvantages described below.

Accordingly, an aspect of the present disclosure is to provide anelectronic device equipped with a display means, which may identify auser input sensed when the electronic device is mounted on a wearabledevice, identify an object related to the user input among objectsdisplayed, determine that the object is intended to be selected by theuser, and output the object.

According to an aspect of the present disclosure, an electronic deviceis provided, which includes a display; a sensor; and at least oneprocessor configured to display a stereoscopic screen including aleft-eye screen and a right-eye screen through the display, move apointer object in the stereoscopic screen, based on a movement of theelectronic device sensed by the sensor, identify whether the pointerobject is positioned at a first position for a predetermined time in thestereoscopic screen, based on information obtained from the sensor, andbased on identifying that the pointer object is positioned at the firstposition for the predetermined time, move at least one object includedin the stereoscopic screen into the first position at which the pointerobject is positioned.

According to another aspect of the present disclosure, a method isprovided for controlling an object displayed through an electronicdevice. The method includes displaying a stereoscopic screen including aleft-eye screen and a right-eye screen through a display of theelectronic device; moving a pointer object in the stereoscopic screen,based on a movement of the electronic device sensed by a sensor of theelectronic device; identifying whether the pointer object is positionedat a first position for a predetermined time in the stereoscopic screen,based on information obtained from the sensor; and based on identifyingthat the pointer object is positioned at the first position for thepredetermined time, moving at least one object included in thestereoscopic screen into the first position at which the pointer objectis positioned.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of the presentdisclosure will become better understood by reference to the followingdetailed description when considered in connection with the accompanyingdrawings, in which:

FIG. 1 illustrates a configuration of an electronic device, according toan embodiment of the present disclosure;

FIG. 2 illustrates a method of controlling an object displayed by anelectronic device, according to an embodiment of the present disclosure;

FIG. 3 illustrates a method of determining at least one object among aplurality of objects according to at least one reference by anelectronic device, according to an embodiment of the present disclosure;

FIGS. 4A and 4B illustrate an example of a wearable device, according toan embodiment of the present disclosure;

FIG. 5 illustrates an example in which a user wears an electronicdevice-mounted wearable device, according to an embodiment of thepresent disclosure;

FIG. 6 illustrates a screen mode of an electronic device, according toan embodiment of the present disclosure;

FIG. 7 illustrates an example of a display screen of an electronicdevice when the electronic device is mounted on a wearable device,according to an embodiment of the present disclosure;

FIGS. 8A to 8D illustrate screens displayed through an electronicdevice, according to an embodiment of the present disclosure;

FIG. 9 illustrates a screen displayed through an electronic device,according to an embodiment of the present disclosure;

FIG. 10 illustrates a screen displayed in an HMD mode, according to anembodiment of the present disclosure;

FIG. 11 illustrates when an object is moved, according to an embodimentof the present disclosure;

FIG. 12 illustrates when an object is moved, according to an embodimentof the present disclosure;

FIG. 13 illustrates when an object is moved, according to an embodimentof the present disclosure;

FIG. 14 illustrates a method of moving an object displayed through anelectronic device, according to an embodiment of the present disclosure;

FIG. 15 illustrates when a displayed object is moved, according to anembodiment of the present disclosure;

FIG. 16 illustrates when a displayed object is moved, according to anembodiment of the present disclosure;

FIG. 17 illustrates an example of an application execution screen wherea displayed object is moved, according to an embodiment of the presentdisclosure;

FIG. 18 illustrates when a displayed object is moved, according to anembodiment of the present disclosure;

FIG. 19 illustrates an example of an application execution screen wherea displayed object is moved, according to an embodiment of the presentdisclosure;

FIG. 20 illustrates an example of an application execution screen wherea displayed object is moved, according to an embodiment of the presentdisclosure;

FIG. 21 illustrates when a displayed object is run, according to anembodiment of the present disclosure;

FIG. 22 illustrates an example network environment, according to anembodiment of the present disclosure;

FIG. 23 illustrates a configuration of an electronic device, accordingto an embodiment of the present disclosure; and

FIG. 24 illustrates an example program module structure, according to anembodiment of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE DISCLOSURE

Embodiments of the present disclosure are described with reference tothe accompanying drawings. However, it should be appreciated that thepresent disclosure is not limited to the embodiments, and all changesand/or equivalents or replacements thereto also pertain to the scope ofthe present disclosure. The same or similar reference denotations may beused to refer to the same or similar elements throughout thespecification and the drawings. Descriptions of well-known functionsand/or configurations will be omitted for the sake of clarity andconciseness.

As used herein, the expressions “have,” “may have,” “include,” or “mayinclude” a feature (e.g., a number, function, operation, or a componentsuch as a part) indicate the existence of the feature and do not excludethe existence of other features.

As used herein, the expressions “A or B,” “at least one of A and/or B,”and “one or more of A and/or B” may include all possible combinations ofA and B. For example, “A or B,” “at least one of A and B,” and “at leastone of A or B” may indicate (1) including at least one A, (2) includingat least one B, or (3) including at least one A and at least one B.

As used herein, the terms “first” and “second” may modify variouscomponents regardless of importance and/or order and are used todistinguish a component from another without limiting the components.For example, a first user device and a second user device may indicatedifferent user devices from each other regardless of the order orimportance of the devices. A first component may be referred to as asecond component, and vice versa without departing from the scope of thepresent disclosure.

It will be understood that when an element, such as a first element, isreferred to as being operatively or communicatively “coupled with/to,”or “connected with/to” another element, such as a second element, thefirst element can be coupled or connected with/to the second elementdirectly or via a third element. In contrast, it will be understood thatwhen the first element is referred to as being “directly coupledwith/to” or “directly connected with/to” the second element, no thirdelement exists between the first and second elements.

As used herein, the expression “configured (or set) to” may beinterchangeably used with the expressions “suitable for,” “having thecapacity to,” “designed to,” “adapted to,” “made to,” or “capable of”depending on circumstances. The expression “configured (or set) to” doesnot essentially indicate “specifically designed in hardware to.” Rather,the expression “configured to” may indicate that a device can perform anoperation together with another device or parts of a device. Forexample, the expression “processor configured (or set) to perform A, B,and C” may indicate a generic-purpose processor, such as a centralprocessing unit (CPU) or application processor that may perform theoperations by executing one or more software programs stored in a memorydevice or a dedicated or embedded processor for performing theoperations.

The terms as used herein are provided merely to describe someembodiments thereof, but not to limit the scope of other embodiments ofthe present disclosure. It is to be understood that the singular forms“a,” “an,” and “the” include plural references unless the contextclearly dictates otherwise. The terms including technical and scientificterms used herein have the same meaning as commonly understood by one ofordinary skill in the art to which the embodiments of the presentdisclosure pertain. It will be further understood that terms, such asthose defined in commonly used dictionaries, should be interpreted ashaving a meaning that is consistent with their meaning in the context ofthe relevant art and will not be interpreted in an idealized or overlyformal sense unless expressly so defined herein. The terms definedherein should not be interpreted to exclude embodiments of the presentdisclosure.

For example, an electronic device according to embodiments of thepresent disclosure may include at least one of a smartphone, a tabletpersonal computer (PC), a mobile phone, a video phone, an e-book reader,a desktop PC, a laptop computer, a netbook computer, a workstation, apersonal digital assistant (PDA), a portable multimedia player (PMP), amotion pictures experts group (MPEG) layer audio 3 (MP3) player, amobile medical device, a camera, or a wearable device.

According to an embodiment of the present disclosure, the wearabledevice may include at least one of an accessory-type device, such as awatch, a ring, a bracelet, an anklet, a necklace, glasses, contactlenses, or an HMD, a fabric- or clothes-integrated device such aselectronic clothes, a body attaching-type device such as a skin pad ortattoo, or a body implantable device, such as an implantable circuit.

According to an embodiment of the present disclosure, the electronicdevice may be a home appliance such as a television, a digital videodisk (DVD) player, an audio player, a refrigerator, an air conditioner,a cleaner, an oven, a microwave oven, a washer, a dryer, an air cleaner,a set-top box, a home automation control panel, a security controlpanel, a TV box (e.g., Samsung HomeSync™, Apple TV™, or Google TV™), agaming console (Xbox™, PlayStation™), an electronic dictionary, anelectronic key, a camcorder, or an electronic picture frame.

According to an embodiment of the present disclosure, examples of theelectronic device may include at least one of various medical devicesincluding diverse portable medical measuring devices such as a bloodsugar measuring device, a heartbeat measuring device, or a bodytemperature measuring device, a magnetic resource angiography (MRA)device, a magnetic resource imaging (MRI) device, a computed tomography(CT) device, an imaging or an ultrasonic device, a navigation device, aglobal navigation satellite system (GNSS) receiver, an event datarecorder (EDR), a flight data recorder (FDR), an automotive infotainmentdevice, a sailing electronic device such as a sailing navigation deviceor a gyro compass, avionics, security devices, vehicular head units,industrial or home robots, automatic teller machines (ATMs), point ofsales (POS) devices, or Internet of Things devices such as a light bulb,various sensors, an electric or gas meter, a sprinkler, a fire alarm, athermostat, a street light, a toaster, fitness equipment, a hot watertank, a heater, or a boiler.

According to embodiments of the disclosure, examples of the electronicdevice may be at least one of part of a piece of furniture orbuilding/structure, an electronic board, an electronic signaturereceiving device, a projector, or various measurement devices such asfor measuring water, electricity, gas, or electromagnetic waves. Theelectronic device may be one or a combination of the above-listeddevices, and may be a flexible electronic device. The electronic devicesdisclosed herein are not limited to the above-listed devices, and mayinclude new electronic devices depending on the development of newtechnology.

According to an embodiment of the present disclosure, an object may bean indication outputted on a display through a stereoscopic image, andwhen a particular object is selected, an operation previously set forthe selected object may be performed on the electronic device.

According to an embodiment of the present disclosure, a pointer objectmay be an object to indicate a particular object. When a preset userinput is entered with the pointer object positioned in an area of afirst object, an operation related to the first object may be performedon the electronic device.

According to an embodiment of the present disclosure, a movement of theelectronic device may include an angle of the movement of the electronicdevice that is based on a preset reference position. For example, theuser may wear the electronic device-mounted wearable device on his/herface and may move his/her face angle up, down, left, or right, and theelectronic device may determine the movement as a movement of theelectronic device.

According to an embodiment of the present disclosure, a movement of afirst object to a second object may indicate that the first object goesinto an area of the second object related to the first object among aplurality of objects, and the first object and the second object may bedisplayed overlapping each other.

Hereinafter, an electronic device, wearable device, and method forcontrolling an object displayed through the electronic device accordingto embodiments of the present disclosure are described with reference tothe accompanying drawings. As used herein, the term “user” may denote ahuman or an artificial intelligence electronic device using theelectronic device.

According to embodiments of the present disclosure, by the electronicdevice, wearable device, and method for controlling objects displayedthrough the electronic device, movements of a plurality of objects maybe controlled corresponding to a sensed movement of the electronicdevice, and one of a first object and a second object related to thefirst object may be moved to the position of the other object, enablinga movement of an object to be controlled with minimal user movement.

FIG. 1 illustrates a configuration of an electronic device according toan embodiment of the present disclosure.

Referring to FIG. 1 , the electronic device 101 may include a display110, a sensor module 120, a processor 130, and a memory 140.

According to an embodiment of the present disclosure, when theelectronic device 101 is mounted on a wearable device 102, theelectronic device 101 may be operated in an HMD mode. When theelectronic device 101 is functionally connected with the wearable device102, an HMD mode screen may be outputted through the electronic device101.

When the electronic device 101 is mounted on the wearable device 102 andoperated in the HMD mode, the screen displayed through the display 110may display two screens respectively corresponding to the user's leftand right eyes.

The display 110 may output a VR screen when the electronic device 101 ismounted on the wearable device 102, such as in the manner of astereoscopic image.

The sensor module 120 may include an acceleration sensor 121 sensing amovement acceleration of the electronic device 101 and a touch sensor122 sensing a touch input, and the sensor module 120 may further includevarious sensors capable of sensing the movement of the electronic device101 or another electronic device. For example, according to a movementangle of the electronic device 101, objects displayed in a VR space maybe selected or a pointer object indicating that a particular object isselectable may be moved.

According to an embodiment of the present disclosure, the electronicdevice 101 may set a reference position in the VR space, and the sensormodule 120 may sense an angle at which the electronic device 101 ismoved based on the set reference position. For example, as the referenceposition, a position value corresponding to an angle of the electronicdevice 101 when the user views the front of the worn electronic device102 may be set. Other position values as designated by the user may beset as the reference position.

The processor 130 may include a VR screen output unit 131, an objectdisplay controller 132, an object movement controller 133, and an objectselection setting unit 134 and may further include various componentsfor controlling objects displayed based on the determined user movement.

The VR screen output unit 131 may perform control to output a VR screen,such as to output a pointer object for selecting and controlling objectsincluded in the VR screen. The pointer object may perform control tooutput a fixed pointer that has a fixed position on the VR screen, and alocation where an object is snapped or a moving pointer moving as theelectronic device 101 moves.

According to an embodiment of the present disclosure, when theelectronic device 101 is determined to be mounted on the wearable device102, the VR screen output unit 131 may perform control to output astereoscopic image through the display 110 and to output the VR screenunder other circumstances.

The object display controller 132 may perform control to display,through the display 110, a movement of a pointer object or objectincluded in an application execution screen. For example, when aparticular object is set to be selectable as per at least one reference,the object display controller 132 may display the object such that thepointer object overlaps the object.

The object movement controller 133 may control a movement of the objectdisplayed through the VR screen based on a movement angle of theelectronic device 101 sensed through the sensor module 120.

According to an embodiment of the present disclosure, the objectmovement controller 133 may control the movement of the object furtherconsidering a sensor input identified from the wearable device 102.

When the pointer object is rendered to have a fixed position accordingto the movement of the electronic device 101 sensed through the sensormodule 120, and a first object of the VR screen is included in an areaof the pointer object having the fixed position, the object selectionsetting unit 134 may determine that the first object is pointed out.

According to an embodiment of the present disclosure, when the pointerobject is positioned to overlap a particular object, the objectselection setting unit 134 may set such that the overlapping object maybe selected. For example, upon identifying a preset input, such as agesture input or button selection for the object set to be selectable,the object selection setting unit 134 may perform control to run anoperation related to the object overlapping the pointer object.

According to an embodiment of the present disclosure, the objectselection setting unit 134 may determine an object corresponding to amovement of the electronic device 101 among objects outputted on the VRscreen as per at least one reference, which may include the position ofeach object and the pointer object, the importance of the objects, andthe attribute of connection between the objects or input method, or mayfurther include various references for determining an object intended tobe selected by the user among from the plurality of objects.

The memory 140 includes pointer control information 141 and anapplication program 142, and may further include information fordetermining a movement of the electronic device 101 and determining anobject to be selected based on the movement of the electronic device101.

The pointer control information 141 may include information for movingobjects outputted in a stereoscopic image into another object. Forexample, the information may include information about the position ofeach object and the pointer object, the importance of the objects, andthe attribute of connection between the objects or input method, and mayfurther include information for moving any one of the plurality ofobjects into the position of another object.

The application program 142 may include information related to anapplication for running on a stereoscopic image screen.

The wearable device 102 may include a device for outputting an HMD modescreen. For example, the electronic device 101 and the wearable device102 may be configured in their respective devices or both may beconfigured in a single device.

According to an embodiment of the present disclosure, upon identifyingthat the electronic device 101 and the wearable device 102 arefunctionally or electrically coupled together, the display 110 of theelectronic device 101 may perform control to output an HMD mode screen.

FIG. 2 illustrates a method of controlling an object displayed by anelectronic device, according to an embodiment of the present disclosure.

Referring to FIG. 2 , the electronic device in step 210 may sense amovement of the electronic device, such as through a device functionallyconnected with the electronic device or through an acceleration sensoror gyro sensor in the electronic device.

In step 220, the electronic device may move a first object in thestereoscopic image from a first position to a second positioncorresponding to the sensed movement. For example, the first object maybe moved up, down, left, or right in the stereoscopic image,corresponding to an angle of the electronic device with respect to apredetermined position.

In step 230, the electronic device may move one of the first object andsecond object to the position of the other object. For example, theelectronic device may determine whether to move the first object or thesecond object according to the type of the running application orwhether the object is moved.

According to an embodiment of the present disclosure, when the secondobject is a fixed object on the screen, the first object movingcorresponding to the movement of the electronic device may be moved tothe position of the second object.

According to an embodiment of the present disclosure, when the secondobject is a progress bar object moving in continuous actions on thescreen, the second object may be moved into the position of the firstobject corresponding to the movement of the electronic device.

FIG. 3 illustrates a method of determining at least one object among aplurality of objects according to at least one reference by anelectronic device, according to an embodiment of the present disclosure.

Objects and a pointer object between the objects are displayed on ascreen outputted from the electronic device, and the pointer objectpositioned between the objects may be moved into an area of any one ofthe objects according to at least one reference for the objects,according to embodiments of the present disclosure.

The steps of FIG. 3 described below may be performed in a differentorder or at least one configuration may be varied or omitted.

Referring to FIG. 3 , in step 311, the electronic device may determinewhether the position of the pointer object between the objects can bedetermined.

When the position between the objects is determined to be determinablein step 311, the electronic device in step 321 may identify an objectpositioned closest to the pointer object among the objects.

When the position between the objects is determined not to bedeterminable in step 311, the electronic device in step 312 maydetermine whether an importance between the objects is determinable,such as based on one of the count of running a corresponding object andrelevance with the user.

When the importance between the objects is determined to be determinablein step 312, the electronic device in step 322 may identify the higherin importance of the objects.

When the importance between the objects is determined not to bedeterminable in step 312, the electronic device 313 may determinewhether a movement direction of the pointer object is determinable.

When the movement direction of the pointer object is determined to bedeterminable in step 313, the electronic device in step 323 may identifyan object positioned in the direction along which the pointer objectmoves among the objects.

When the movement direction of the pointer object is determined not tobe determinable in step 313, the electronic device in step 314 maydetermine whether a connection attribute between the objects isdeterminable. For example, the connection attribute between the objectsmay be related information about an object selected earlier than theobjects and each of the objects and may be considered to identify anobject related to the pointer object among the objects.

When the connection attribute between the objects is determined to bedeterminable in step 314, the electronic device in step 324 may identifyan object related to the previously selected object among the objects.

When the connection attribute between the objects is determined not tobe determinable in step 314, the electronic device in step 315 maydetermine whether a user input, other than the user movement forcontrolling the pointer object is entered. For example, the user inputmay include the user's voice data, gesture input such as touch,biological information or other inputs related to the user that areentered through the electronic device.

When entry of the other user input is determined to be determinable instep 315, the electronic device in step 325 may identify an objectrelated to the entered user input among the objects.

As at least one of the above-described operations 321, 322, 323, 324,and 325 is performed, the electronic device in step 330 may move thepointer object into an area of the object identified through theabove-described operations.

FIGS. 4A and 4B illustrate an example of a wearable device, according toan embodiment of the present disclosure.

Referring to FIGS. 4A and 4B, the wearable device 410 may be theelectronic device 102 of FIG. 1 , for example. The wearable device 410may provide only functions of a mount or support with no communicationfunctionality with the electronic device 420. The wearable device 410may include a body 401 and a cover 402.

When the electronic device 420 is mounted on the wearable device 410,the cover 402 may cover the edge of a rear surface of the electronicdevice 420 to be fastened to the wearable device 410 so that theelectronic device 420 may remain in a mounted position. The wearabledevice 410 may have a support available for the user to wear thewearable device 410 on his/her head.

The wearable device 410 may have lenses respectively corresponding to aneye of the user (wearer). The user may view the display of theelectronic device 420 through the lenses with the electronic device 420mounted on the wearable device 410. The wearable device 410 may have amechanical or instrumental structure allowing the electronic device 420to be mounted on the wearable device 410 as shown.

According to an embodiment of the present disclosure, the wearabledevice 410 may include at least one of the components of the electronicdevice 101 shown in FIG. 1 , such as a touch panel, a button, a wheelkey, and a touch pad. The touch panel may receive the user's touchinput, such as the user's touch on the touch panel or a hovering inputnear the touch panel. When the electronic device 420 is mounted, thewearable device 410 may connect with the electronic device 420 via aninterface such as USB, to communicate with the electronic device 420.

The electronic device 420 may control a function corresponding to aninput received from the wearable device 410 in response to the input,such as adjusting the sound volume or controlling the screen (e.g., animage playing screen of the HMD mode) in response to the received input.For example, when the electronic device 420 is mounted on the wearabledevice 410, an external device connecting unit of the electronic device420 may be electrically connected with the connecting unit of thewearable device 410 to enable communication between the devices.

FIG. 5 illustrates an example in which a user wears an electronicdevice-mounted wearable device, according to an embodiment of thepresent disclosure.

In FIG. 5 , the user may control the functions of the electronic device520 or of the wearable device 510 through the touch pad 530 provided ina side surface of the wearable device 510.

Referring to FIGS. 4A, 4B, and 5 , the electronic devices 420 and 520may be the electronic device 101 of FIG. 1 , such as smartphones with adisplay means. The wearable devices 410 and 510 may be the electronicdevice 101 or wearable device 102 of FIG. 1 or may be instrumental,wearable mounts.

As shown in FIGS. 4A, 4B, and 5 , the electronic devices 420 and 520 maybe mounted on the wearable devices 410 and 510 so that the display,i.e., the touchscreen 530 faces the user's face. The user may fasten theelectronic devices 420 and 520 to the wearable devices 410 and 510 bygetting closer to the covers of the wearable devices 410 and 510. InFIG. 5 , the user may wear the electronic device 520—mounted wearabledevice 510 on his/her head, and may view the screen of the display ofthe electronic device 520 through the lenses of the wearable device 510.

FIG. 6 illustrates a screen mode of an electronic device, according toan embodiment of the present disclosure.

Referring to FIG. 6 , the electronic device 600 may be the electronicdevice 420 or 520 as shown in FIGS. 4A, 4B, and 5 . When operating inthe normal mode, the electronic device 600 may display one operationscreen 610 as shown in the upper screen of FIG. 6 .

According to an embodiment of the present disclosure, when mounted on awearable device, the electronic device 600 may be operated in the HMDmode as shown in the lower screen of FIG. 6 . When the electronic device600 is operated in the HMD mode, a screen 620 a corresponding to theuser's left eye and another screen 620 b corresponding to the user'sright eye may be distinctively displayed. In the screen of the HMD mode,one image may be separately displayed as two images 620 a and 620 b.

According to an embodiment of the present disclosure, when theelectronic device 600 is mounted on the wearable device and operated inthe HMD mode, the display area or display position of the screencorresponding to the HMD mode (the screen 620 a corresponding to theleft eye and the screen 620 b corresponding to the right eye) may bedetermined.

FIG. 7 illustrates an example of a display screen of an electronicdevice when the electronic device is mounted on a wearable device,according to an embodiment of the present disclosure.

Referring to FIG. 7 , the user 701 may wear an electronic device702—mounted wearable device 703. For example, an HMD mode screen 721 maybe outputted on the display of the electronic device 702.

According to an embodiment of the present disclosure, the HMD modescreen 721 is configured in a three-dimension (3D) stereoscopic image,and objects corresponding to points of a movement of the electronicdevice 702 among objects of the HMD mode screen 721 may be outputtedthrough the electronic device 702. For example, when the user 701 liftshis/her face up, the electronic device 702 may move upwards accordingly,and a screen and objects positioned over a reference position of the HMDmode screen 721 may be outputted through the electronic device 702.

According to an embodiment of the present disclosure, when a pointerobject is outputted on the HMD mode screen 721, the electronic device702 may control the pointer object according to the movement of theelectronic device 702.

FIGS. 8A to 8D illustrate screens displayed through an electronicdevice, according to an embodiment of the present disclosure.

Referring to FIG. 8A, the electronic device may output a screen 800 aincluding thumbnail images 810 a for at least one webpage or controlobjects 820 a for accessing a particular webpage. For example, when oneof the thumbnail images is selected through the screen 800 a, theelectronic device may access the website corresponding to the selectedthumbnail image 810 a.

According to an embodiment of the present disclosure, when one of thecontrol objects 820 a is selected, the electronic device may perform anoperation such as move back or go to home screen, for controlling thepreviously selected object (e.g., the thumbnail image 810 a).

Referring to FIG. 8B, the electronic device may output an executionscreen 800 b of a program (e.g., “quick access”) for running at leastone application. For example, the at least one application may be run onthe electronic device for a preset time or may be designated accordingto the count of execution.

According to an embodiment of the present disclosure, the executionscreen may include thumbnail images 810 b for the at least oneapplication. For example, when one of the thumbnail images 810 b isselected, the electronic device may run an application corresponding tothe selected thumbnail image, or may run a website corresponding to theselected thumbnail image or a new website.

Referring to FIG. 8C, the electronic device may output an executionscreen 800 c of a media playing application, which may include an area810 c where a playing media file is outputted and an area 820 c forcontrolling the playing of the media file.

Referring to FIG. 8D, the screen 800 d displayed through the electronicdevice may include a thumbnail image 810 d of at least one program andan object related to a particular program. For example, the at least oneprogram may include a program (e.g., “quick access”) for running atleast one application, a web browser (e.g., VR Internet), or bookmarksand may further include a thumbnail image of a program designatedaccording to the count of running on the electronic device for a presettime.

According to an embodiment of the present disclosure, when a user inputis entered or pointer object is overlapped for an area (e.g., thethumbnail image 810 d) of any one program, the electronic device may runa program related to the selected thumbnail image 810 d.

FIG. 9 illustrates a screen displayed through an electronic device,according to an embodiment of the present disclosure.

Referring to FIG. 9 , the electronic device may output an HMD modescreen 900 which may include an object such as a first webpage 911,second webpage 912, and third webpage 913 of an execution screen of atleast one application and a button object 902 and pointer object 901performing a pre-designated operation.

According to an embodiment of the present disclosure, the pointer object901 may be moved as the electronic device moves, and when the movingpointer object 901 overlaps one of the outputted objects, a control maybe performed such that the object overlapping the pointer object 901 ishighlighted. For example, a control may be performed such that otherindications that the pointer object 901 is overlapped, such as byenlarging or brightening, may be outputted through the particular objecton the HMD mode screen 900.

According to an embodiment of the present disclosure, when the pointerobject 901 is positioned between the second webpage 912 and the buttonobject 902, the electronic device may move the pointer object 901 intothe position of one of the second webpage 912 and the button object 902.For example, the electronic device may move the pointer object 901 intothe position of one of the objects based on the distance between thepointer object 901 and each of the second webpage 912 and button object902 and the movement direction of the pointer object 901.

FIG. 10 illustrates a screen displayed in an HMD mode, according to anembodiment of the present disclosure.

Referring to FIG. 10 , the HMD mode screen 1000 may include at least oneobject, such as a first object 1011 and second object 1012 forperforming a pre-designated operation and a pointer object 1001 forindicating the at least one object.

According to an embodiment of the present disclosure, the movement ofthe pointer object 1001 may be controlled according to the movement ofthe user wearing the electronic device outputting the HMD mode screen1000.

According to an embodiment of the present disclosure, when the pointerobject 1001 is moved into an area between the first object 1011 and thesecond object 1012, the electronic device may move the pointer object1001 to overlap one of the first object 1011 or the second object 1012according to at least one reference. For example, when the distancebetween the pointer object 1001 and the first object 1011 is greaterthan the distance between the pointer object 1001 and the second object1012, the electronic device may move the pointer object 1001 into theposition of the second object 1012.

FIG. 11 illustrates when an object is moved, according to an embodimentof the present disclosure.

Referring to FIG. 11 , the HMD mode screen 1100 may include at least oneobject, such as a first object (e.g., ‘A’) 1101 and second object (e.g.,‘B’) 1102 for performing a pre-designated operation and a pointer object(e.g., ‘P’) 1103 for indicating the at least one object.

According to an embodiment of the present disclosure, the movement ofthe pointer object 1103 may be controlled according to the movement ofthe user wearing the electronic device outputting the HMD mode screen1100.

When the pointer object 1103 is positioned between the first object 1101and the second object 1102 for a preset time such as 3 seconds, theelectronic device may move the pointer object 1103 to overlap one of thefirst object 1101 or the second object 1102 according to the movementdirection of the pointer object 1103. For example, when the pointerobject 1103 moves in a direction from the first object 1101 to thesecond object 1102, the electronic device may move the pointer object1103 into an area where the second object is displayed such that thepointer object 1103 is displayed overlapping the second object 1102.

FIG. 12 illustrates when an object is moved, according to an embodimentof the present disclosure.

Referring to FIG. 12 , the HMD mode screen may include a keypad object1200 for entering a pre-designated character or a pointer object 1201for indicating the at least one object. The keypad object 1200 mayinclude an object (e.g., a “j” button or “i” button) related to at leastone character (e.g., an alphabet character, number, symbol, consonant,or vowel).

According to an embodiment of the present disclosure, the movement ofthe pointer object 1201 may be controlled according to the movement ofthe user wearing the electronic device outputting the HMD mode screen.

According to an embodiment of the present disclosure, when the pointerobject 1201 is positioned between the “j” button and the “i” button forthe preset time, the electronic device may identify an object where thepointer object 1201 was previously positioned and move the pointerobject 1201 to overlap the object where the pointer object 1201 waspreviously positioned. For example, when the pointer object 1201 waspositioned on the “s” button corresponding to a consonant, theelectronic device may move the pointer object 1201 into the area wherethe “i” button is displayed to overlap the “i” button corresponding to avowel.

According to an embodiment of the present disclosure, the keypad object1200 may be set to receive an input of a Korean character. When acharacter showing the end of a sentence, such as “

” or “

,” is entered, the electronic device may move the pointer object 1201into an object corresponding to a period among objects positionedadjacent to the pointer object 1201.

FIG. 13 illustrates when an object is moved, according to an embodimentof the present disclosure.

Referring to FIG. 13 , the HMD mode screen 1300 may include at least oneof a first object (e.g., ‘A’) 1301 and second object (e.g., ‘B’) 1302for performing a pre-designated operation and a pointer object (e.g.,‘P’) 1303 for indicating the at least one object.

According to an embodiment of the present disclosure, the movement ofthe pointer object 1303 may be controlled according to the movement ofthe user wearing the electronic device outputting the HMD mode screen1300.

According to an embodiment of the present disclosure, when the pointerobject 1303 is positioned between the first object 1301 and the secondobject 1302 for the preset time, the electronic device may move thepointer object 1303 to overlap one of the first object 1301 and thesecond object 1302 further considering a user input, such as voice dataor touch input, other than the user's movement.

For example, the electronic device may move the pointer object 1303 intothe area where the second object is displayed further considering theuser's voice data so that the pointer object 1303 is displayedoverlapping the first object 1301 related to information (e.g., acommand for running a gallery application, entry of the name (“B”) ofthe second object 1302 or “right-hand”) contained in the voice data.

For example, when a touch input sensed through the electronic device orwearable device is identified to be a pre-designated input (e.g., aswipe input to the left), the electronic device may move the pointerobject 1303 into the object corresponding to the designated input.

FIG. 14 illustrates a method of moving an object displayed through anelectronic device, according to an embodiment of the present disclosure.

Referring to FIG. 14 , in step 1410, the electronic device may determinewhether a pointer object is positioned within a preset area for aparticular object. For example, the preset area may include at least aportion of the particular object.

When the pointer object is determined not to be positioned within thepreset area in step 1410, the electronic device in step 1420 maydetermine whether the pointer object is positioned in an area of anindependent object, such as at least a portion of an area other than thearea of the particular object in the entire display area.

When the pointer object is determined not to be positioned in the areaof the independent object in step 1420, the electronic device in step1430 may determine whether an object moving in continuous actions isidentified. For example, the object moving in continuous actions mayinclude an indicator on a progress bar.

When the pointer object is determined to be positioned in the presetarea or independent area or an object moving in continuous actions instep 1410, 1420, or 1430, the electronic device in step 1440 may movethe pointer object into the position on which the user's eyes focus. Forexample, the position on which the user's eyes focus may correspond towhere the user's eyes remain and an angle at which the electronic devicemoves up, down, left, or right with respect to a predesignated referenceposition.

FIG. 15 illustrates when a displayed object is moved, according to anembodiment of the present disclosure.

Referring to FIG. 15 , the electronic device may output an executionscreen 1500 of a media playing application, which may include an areawhere the media file plays and a video controller button 1510 forcontrolling the playing of the media file. For example, the videocontroller button 1510 may be outputted as the pointer object 1501 has afixed position in a preset area of the screen 1500 for a predeterminedtime.

According to an embodiment of the present disclosure, when the videocontroller button 1510 is outputted, the electronic device may move thepointer object 1501 into the position of the video controller button1510.

According to an embodiment of the present disclosure, the electronicdevice may control the movement of displayed objects to perform variousoperations related to an object with minimized user movement.

FIG. 16 illustrates when a displayed object is moved, according to anembodiment of the present disclosure.

Referring to FIG. 16 , the electronic device may output an executionscreen 1600 of an application managing a list of objects set asbookmarks by the user in the HMD mode. For example, the execution screen1600 may include thumbnail images 1610 respectively corresponding to thebookmarks and a control button area 1620 for controlling the thumbnailimages. The control button area 1620 may include a “SELECT ALL” button1621 for selecting all the thumbnail images, a “CANCEL” button 1622 forcanceling the selection of a thumbnail image, and a “DONE” button 1623for ensuring that a thumbnail image is selected.

According to an embodiment of the present disclosure, when a presetoperation is run through the execution screen, the electronic device maymove the pointer object 1601 into the control button area 1620. Forexample, when at least one object (e.g., 1611, 1612, or 1613) isselected from the bookmarks, and the electronic device moves at an angleallowing the electronic device to face in a lower direction, theelectronic device may move the pointer object 1601 into the position ofthe CANCEL button 1622 or DONE button 1623.

According to an embodiment of the present disclosure, the electronicdevice may move the pointer object 1601 into the area of one of theCANCEL button 1622 or DONE button 1623 or an area between the CANCELbutton 1622 and the DONE button 1623, considering at least one referenceposition of each of the buttons on the execution screen 1600.

According to an embodiment of the present disclosure, upon failing tosense the user's movement for a predetermined time through the executionscreen, the electronic device may move the pointer object 1601 into theposition of the SELECT ALL button 1621.

FIG. 17 illustrates an example of an application execution screen wherea displayed object is moved, according to an embodiment of the presentdisclosure.

Referring to FIG. 17 , the electronic device 1700 may output anexecution screen of a media playing application, such as a video file.The execution screen may include an area 1710 where the video file playsand a control button area 1720 for controlling the playing of the videofile.

According to an embodiment of the present disclosure, the control buttonarea 1720 may include a progress bar area 1720 having a first object1721 indicating a time played and a pointer object 1701 moving as theelectronic device moves.

According to an embodiment of the present disclosure, the first object1721 of the progress bar area 1720 may be moved in a predeterminedaction as the media file plays and may move into the position of thepointer object 1701 as the electronic device moves.

The user may fasten the pointer object 1701 to an area corresponding toa first time for a predetermined time. For example, the electronicdevice may identify the time when the movement of the pointer object1701 is fastened and move the first object 1721 into the areacorresponding to the identified first time.

When a predetermined time passes after the first object 1721 has movedinto the area corresponding to the first time, the video file may beplayed from the section corresponding to the first time through the area1710 where the video file plays.

FIG. 18 illustrates when a displayed object is moved, according to anembodiment of the present disclosure.

Referring to FIG. 18 , the electronic device may output a screen 1800for setting functions of the display. For example, the setting screen1800 may include a button 1821 for setting brightness.

When the brightness setting button 1821 is selected, the electronicdevice may output a progress bar 1810 for adjusting brightness. Theprogress bar 1810 may include a first object 1811 indicating abrightness level and a pointer object 1801 moving as the electronicdevice moves.

According to an embodiment of the present disclosure, the first object1811 of the progress bar area 1810 may be moved into the position of thepointer object 1801 as the electronic device moves.

The user may fasten the pointer object 1801 into a position 1812corresponding to a first brightness level, such as level 5, for apredetermined time. For example, the electronic device may identify thebrightness level at which the movement of the pointer object 1801 isfastened and may move the first object 1811 into the position 1812corresponding to the identified first brightness level.

When a predetermined time elapses after the first object 1811 has movedinto the position 1812 corresponding to the first brightness level, theelectronic device may control the display so that the brightness of thedisplay is set to the first brightness level.

FIG. 19 illustrates an example of an application execution screen wherea displayed object is moved, according to an embodiment of the presentdisclosure.

Referring to FIG. 19 , the electronic device may output an executionscreen 1900 of a web browser application. For example, the executionscreen 1900 may include at least one webpage (e.g., a first webpage1911), a button area 1920 for controlling functions related to theoutputted webpage, and a pointer object 1901 moving as the user moves.

According to an embodiment of the present disclosure, when the pointerobject 1901 remains on at least a portion of the area of the firstwebpage 1911 for a predetermined time, the electronic device mayidentify a left-right scroll button 1921 designated for where thepointer object 1901 has a fixed position for a predetermined time, andmay move the left-right scroll button 1921 into the pointer object 1901.

For example, the user may designate an application displaying a list ofthumbnail images or webpages, for an application where a preset objectmoves into the position of the pointer object 1901, and other objectsmay be positioned at left and right sides to implement variousapplications requiring the left-right scroll button 1921. The electronicdevice may move an object which is positioned in line with the pointerobject 1911 in an upward, downward, right, or left direction, into theposition of the pointer object 1911.

According to an embodiment of the present disclosure, the electronicdevice may perform control to output a prior or subsequent webpage withrespect to the first webpage 1911 according to the left or rightmovement of the electronic device, with the left-right scroll button1921 relocated into the pointer object 1901.

FIG. 20 illustrates an example of an application execution screen wherea displayed object is moved, according to an embodiment of the presentdisclosure.

Referring to FIG. 20 , the electronic device may output, through adisplayed screen 2000, an execution screen (e.g., a first webpage 2010)of a web browser application having run for a predetermined time or apointer object 2001 moving as the electronic device moves.

According to an embodiment of the present disclosure, when the pointerobject 2001 remains in a designated area in the area of the firstwebpage 2010 for a predetermined time, the electronic device may outputan upper-lower scroll button 2011 and move the pointer object 2001 intothe designated object. For example, the designated area in the area ofthe first webpage 2010 may be a character data-free portion of the areaof the first webpage 2010 or an advertisement data-displayed portion ofthe area of the first webpage 2010.

According to an embodiment of the present disclosure, while the pointerobject 2001 is positioned on the upper-lower scroll button 2011, theelectronic device may perform control to output an upper or lowerportion of the first webpage 2010 as the electronic device moves up ordown.

FIG. 21 illustrates when a displayed object is run, according to anembodiment of the present disclosure.

Referring to FIG. 21 , the electronic device may output a stereoscopicimage 2110 through a display. For example, a pointer object 2101 movingas the electronic device moves may be outputted on the stereoscopicimage 2110.

The electronic device may identify the position of the pointer object2101 and trigger a timer. For example, the timer may determine whetherthe pointer object 2101 has a fixed position for a predetermined time.

According to an embodiment of the present disclosure, when the pointerobject 2101 has a fixed position in the area of a first object 2111, theelectronic device may perform a function related to the first object2111. For example, a function (e.g., “shots”) showing pictures capturedfor the first object may be designated, and the electronic device mayoutput a screen 2120 including the captured pictures, as shown in thelower image of FIG. 21 .

FIG. 22 illustrates an example network environment, according to anembodiment of the present disclosure.

Referring to FIG. 22 , the network environment 2200 may include anelectronic device 2201, at least one of a first electronic device 2202and a second electronic device 2204, and a server 2206, and each of thecomponents may be connected with the electronic device 2201 through anetwork 2262 or through a communication interface 2270 of the electronicdevice 2201.

The electronic device 2201 may include a bus 2210, a processor 2220, amemory 2230, an input/output interface 2250, a display 2260, and thecommunication interface 2270. In some embodiments, the electronic device2201 may exclude at least one of the components or may add anothercomponent.

The bus 2210 may include a circuit for connecting the components 2220 to2270 with one another and transferring communications (e.g., controlmessages and/or data) between the components.

The processing module 2220 may include one or more of a centralprocessing unit (CPU), an application processor (AP), and acommunication processor (CP). The processor 2220 may perform control onat least one of the other components of the electronic device 2201,and/or perform an operation or data processing relating tocommunication.

The memory 2230 may include a volatile and/or non-volatile memory. Forexample, the memory 2230 may store commands or data related to at leastone other component of the electronic device 2201. According to anembodiment of the present disclosure, the memory 2230 may store softwareand/or a program 2240 including a kernel 2241, middleware 2243, anapplication programming interface (API) 2245, and/or applications 2247.At least a portion of the kernel 2241, middleware 2243, or API 2245 maybe denoted an operating system (OS).

For example, the kernel 2241 may control or manage system resources(e.g., the bus 2210, processor 2220, or a memory 2230) used to performoperations or functions implemented in other programs. The kernel 2241may provide an interface that allows the middleware 2243, the API 2245,or the applications 2247 to access the individual components of theelectronic device 2201 to control or manage the system resources.

The middleware 2243 may function as a relay to allow the API 2245 or theapplications 2247 to communicate data with the kernel 2241, for example.

The middleware 2243 may process one or more task requests received fromthe applications 2247 in order of priority. For example, the middleware2243 may assign at least one of the applications 2247 with priority ofusing system resources of at least one electronic device 2201. Forexample, the middleware 2243 may perform scheduling or load balancing onthe one or more task requests by processing the one or more taskrequests according to the priority assigned to the at least oneapplication program 147.

The API 2245 enables the applications 2247 to control functions providedfrom the kernel 2241 or the middleware 2243. For example, the API 133may include at least one interface or function for filing control,window control, image processing or text control.

The input/output interface 2250 may serve as an interface that maytransfer commands or data input from a user or other external devices toother component(s) of the electronic device 2201, and may outputcommands or data received from other component(s) of the electronicdevice 2201 to the user or the other external device.

The display 2260 may include a liquid crystal display (LCD), a lightemitting diode (LED) display, an organic light emitting diode (OLED)display, or a microelectromechanical systems (MEMS) display, or anelectronic paper display. The display 2260 may display various contentssuch as text, images, videos, icons, and symbols to the user. Thedisplay 2260 may include a touchscreen and may receive a touch, gesture,proximity or hovering input using an electronic pen or a body portion ofthe user.

For example, the communication module 2270 may set up communicationbetween the electronic device 2201 and an external device (e.g., a firstelectronic device 2202, a second electronic device 2204, or a server2206). For example, the communication module 2270 may be connected withthe network 2262 through wireless or wired communication to communicatewith the external electronic device.

The wireless communication may use at least one of long term evolution(LTE), long term evolution-advanced (LTE-A), code division multipleaccess (CDMA), wideband code division multiple access (WCDMA), universalmobile telecommunication system (UMTS), wireless broadband (WiBro), orglobal system for mobile communication (GSM), as a cellularcommunication protocol, and may include short-range communication 2264,such as wireless fidelity (Wi-Fi), Bluetooth®, near-field communication(NFC), or global navigation satellite system (GNSS), such as globalpositioning system (GPS), Beidou navigation satellite system (Beidou) orGalileo, or the European global satellite-based navigation system. Theterms “GPS” and “GNSS” may be interchangeably used herein. The wiredconnection may include at least one of universal serial bus (USB), highdefinition multimedia interface (HDMI), recommended standard (RS)-232,or plain old telephone service (POTS). The network 2262 may include atleast one of communication networks, such as a computer network (e.g.,local area network (LAN) or wide area network (WAN)), the Internet, or atelephone network.

The first and second external electronic devices 2202 and 2204 each maybe the same type as or a different type than the electronic device 2201.According to an embodiment of the present disclosure, the server 2206may include a group of one or more servers. All or some of operationsexecuted on the electronic device 2201 may be executed on another ormultiple other electronic devices. When the electronic device 2201should perform some function or service automatically or at a request,the electronic device 2201, instead of executing the function or serviceby itself or additionally, may request another device to perform atleast some functions associated therewith. The other electronic device,such as electronic devices 2202 and 2204 or server 2206, may execute therequested functions or additional functions and transfer a result of theexecution to the electronic device 2201. The electronic device 2201 mayprovide a requested function or service by processing the receivedresult as it is or additionally. To that end, a cloud computing,distributed computing, or client-server computing technique may be used,for example.

FIG. 23 illustrates a configuration of an electronic device, accordingto an embodiment of the present disclosure.

Referring to FIG. 23 , the electronic device 2301 may include all orpart of the configuration of the electronic device 2201 shown in FIG. 22. The electronic device 2301 may include one or more applicationprocessors (APs) 2310, a communication module 2320, a subscriberidentification module (SIM) 2324, a memory 2330, a sensor module 2340,an input module 2350, a display 2360, an interface 2370, an audio module2380, a camera module 2391, a power management module 2395, a battery2396, an indicator 2397, and a motor 2398.

The processor 2310 may control multiple hardware and software componentsconnected to the processor 2310 by running an operating system orapplication programs, and may process and compute various data. Theprocessor 2310 may be implemented in a system on chip (SoC). Accordingto an embodiment of the present disclosure, the processor 2310 mayfurther include a graphic processing unit (GPU) and/or an image signalprocessor, and may include at least some of the components shown in FIG.23 . The processor 2310 may load a command or data received from anon-volatile memory on a volatile memory, process the command or data,and store various data in the non-volatile memory.

The communication module 2320 may have the same or similar configurationto the communication module 2270 of FIG. 22 . The communication module2320 may include a cellular module 2321, a Wi-Fi module 2323, aBluetooth® module 2325, a GNSS module 2327, an NFC module 2328, and aradio frequency (RF) module 2329.

The cellular module 2321 may provide voice call, video call, text, orInternet services through a communication network. The cellular module2321 may perform identification or authentication on the electronicdevice 2301 in the communication network using the SIM card 2324.According to an embodiment of the present disclosure, the cellularmodule 2321 may perform at least some of the functions providable by theprocessor 2310, and may include a communication processor (CP).

The Wi-Fi module 2323, the Bluetooth (BT) module 2325, the GNSS module2327, or the NFC module 2328 may include a process for processing datacommunicated through the module. At least two of the cellular module2321, the Wi-Fi module 2323, the Bluetooth module 2325, the GNSS module2327, and the NFC module 2328 may be included in a single integratedcircuit (IC) or an IC package.

The RF module 2329 may communicate data, such as RF signals, and mayinclude a transceiver, a power amp module (PAM), a frequency filter, alow noise amplifier (LNA), and an antenna. According to an embodiment ofthe present disclosure, at least one of the cellular module 2321, theWi-Fi module 2323, the Bluetooth module 2325, the GNSS module 2327, andthe NFC module 2328 may communicate RF signals through a separate RFmodule.

The subscriber identification module 2324 may include a card including asubscriber identification module and/or an embedded SIM, and may containunique identification information such as an integrated circuit cardidentifier (ICCID) or subscriber information such as an internationalmobile subscriber identity (IMSI).

The memory 2330 may include an internal memory 2332 and an externalmemory 2334. The internal memory 2332 may include at least one of avolatile memory (e.g., a dynamic random access memory (DRAM), a staticRAM (SRAM), a synchronous dynamic RAM (SDRAM), etc.) or a non-volatilememory (e.g., a one-time programmable read only memory (OTPROM), aprogrammable ROM (PROM), an erasable and programmable ROM (EPROM), anelectrically erasable and programmable ROM (EEPROM), a mask ROM, a flashROM, a flash memory (e.g., a NAND flash, or a NOR flash), a hard drive,or solid state drive (SSD).

The external memory 2334 may include a flash drive, such as a compactflash (CF) memory, a secure digital (SD) memory, a micro-SD memory, amin-SD memory, an extreme digital (xD) memory, a multi-media card (MMC),and a memory Stick™. The external memory 2334 may be functionally and/orphysically connected with the electronic device 2301 via variousinterfaces.

For example, the sensor module 2340 may measure a physical quantity ordetect a motion state of the electronic device 2301, and may convert themeasured or detected information into an electrical signal. The sensormodule 2340 may include a gesture sensor 2340A, a gyro sensor 2340B, anatmospheric pressure sensor 2340C, a magnetic sensor 2340D, anacceleration sensor 2340E, a grip sensor 2340F, a proximity sensor2340G, a color sensor 2340H such as a red-green-blue (RGB) sensor, a biosensor 2340I, a temperature/humidity sensor 2340J, an illuminationsensor 2340K, an ultra violet (UV) sensor 2340L, and a touch sensor2340M. Additionally or alternatively, the sensing module 2340 mayinclude an e-nose sensor, an electromyography (EMG) sensor, anelectroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, aninfrared (IR) sensor, an iris sensor, or a finger print sensor, and mayfurther include a control circuit for controlling at least one or moreof the sensors included in the sensing module. According to anembodiment of the present disclosure, the electronic device 2301 mayfurther include a processor configured to control the sensor module 2340as part of or separately from the processor 2310, and the electronicdevice 2301 may control the sensor module 2340 while the processor 2310is in a sleep mode.

The input unit 2350 may include a touch panel 2352, a (digital) pensensor 2354, a key 2356, and an ultrasonic input device 2358. The touchpanel 2352 may use at least one of capacitive, resistive, infrared, orultrasonic methods, and may further include a control circuit and atactile layer that provides a user with a tactile reaction.

The (digital) pen sensor 2354 may include a part of a touch panel or aseparate sheet for recognition. The key 2356 may include a physicalbutton, optical key or keypad. The ultrasonic input device 2358 maysense an ultrasonic wave generated from an input tool through amicrophone 2388 to identify data corresponding to the sensed ultrasonicwave.

The display 2360 may include a panel 2362, a hologram device 2364, and aprojector 2366. The panel 2362 may have the same or similarconfiguration to the display 2260 of FIG. 22 and may be implemented tobe flexible, transparent, or wearable. The panel 2362 may also beincorporated with the touch panel 2352 in a module. The hologram device2364 may project three dimensional (3D) images (holograms) in the air byusing light interference. The projector 2366 may display an image byprojecting light onto a screen, which may be located inside or outsideof the electronic device 2301. In accordance with an embodiment, thedisplay 2360 may further include a control circuit to control the panel2362, the hologram device 2364, or the projector 2366.

The interface 2370 may include, e.g., a high definition multimediainterface (HDMI) 2372, a USB 2374, an optical interface 2376, or aD-subminiature (D-sub) 2378. The interface 2370 may be included in thecommunication module 2270 shown in FIG. 22 . Additionally oralternatively, the interface 2370 may include a mobile high-definitionlink (MHL) interface, a secure digital (SD) card/multimedia card (MMC)interface, or infrared data association (IrDA) standard interface.

The audio module 2380 may convert a sound into an electric signal orvice versa, for example. At least a part of the audio module 2380 may beincluded in e.g., the input/output interface 2245 as shown in FIG. 22 .The audio module 2380 may process sound information input or outputthrough a speaker 2382, a receiver 2384, an earphone 2386, or themicrophone 2388.

For example, the camera module 2391 may record still images and videos,and may include one or more image sensors (e.g., front and backsensors), a lens, an Image signal processor (ISP), or a flash such as alight-emitting diode (LED) or xenon lamp.

The power manager module 2395 may manage power of the electronic device2301, for example, and may include a power management Integrated circuit(PMIC), a charger IC, or a battery gauge. The PMIC may have a wiredand/or wireless recharging scheme. The wireless charging scheme mayinclude a magnetic resonance scheme, a magnetic induction scheme, or anelectromagnetic wave based scheme, and an additional circuit, such as acoil loop, a resonance circuit, or a rectifier, may be added forwireless charging. The battery gauge may measure an amount of remainingpower of the battery 2396, a voltage, a current, or a temperature whilethe battery 2396 is being charged. The battery 2396 may include arechargeable battery or a solar battery.

The indicator 2397 may indicate a particular state of the electronicdevice 2301 or a part of the electronic device, such as a booting,message, or recharging state. The motor 2398 may convert an electricsignal to a mechanical vibration and may generate a vibrational orhaptic effect. A processing unit for supporting mobile TV, such as a GPUmay be included in the electronic device 2301. The processing unit forsupporting mobile TV may process media data conforming to a standard fordigital multimedia broadcasting (DMB), digital video broadcasting (DVB),or mediaFlo™.

Each of the aforementioned components of the electronic device mayinclude one or more parts, and a name of the part may vary with a typeof the electronic device.

The electronic device in accordance with embodiments of the presentdisclosure may include at least one of the aforementioned components,omit some of the components, or include additional component(s). Some ofthe components may be combined into an entity, but the entity mayperform the same functions as the components.

FIG. 24 illustrates an example program module structure, according to anembodiment of the present disclosure.

The program module 2410 may include an operating system (OS) controllingresources related to the electronic device 2201 and/or variousapplications 2247 driven on the operating system.

The operating system may include Android, iOS, Windows, Symbian, Tizen,or Bada.

The program 2410 may include a kernel 2420, middleware 2430, anapplication programming interface (API) 2460, and/or applications 2470.At least a part of the program module 2410 may be preloaded on theelectronic device or may be downloaded from an external electronicdevice.

The kernel 2420 may include a system resource manager 2421 and/or adevice driver 2423. The system resource manager 2421 may performcontrol, allocation, or recovery of system resources, and may include aprocess managing unit, a memory managing unit, or a file system managingunit. The device driver 2423 may include a display driver, a cameradriver, a Bluetooth driver, a shared memory driver, a USB driver, akeypad driver, a Wi-Fi driver, an audio driver, or an inter-processcommunication (IPC) driver.

The middleware 2430 may provide various functions to the application2470 through the API 2460 so that the application 2470 may efficientlyuse limited system resources in the electronic device or providefunctions jointly required by applications 2470. According to anembodiment of the present disclosure, the middleware 2430 may include atleast one of a runtime library 2435, an application manager 2441, awindow manager 2442, a multimedia manager 2443, a resource manager 2444,a power manager 2445, a database manager 2446, a package manager 2447, aconnectivity manager 2448, a notification manager 2449, a locationmanager 2450, a graphic manager 2451, and a security manager 2452.

The runtime library 2435 may include a library module used by a compilerin order to add a new function through a programming language while theapplication 2470 is being executed. The runtime library 2435 may performinput/output management, memory management, or operation on arithmeticfunctions.

The application manager 2441 may manage the life cycle of at least oneof the applications 2470. The window manager 2442 may manage GUIresources used on the screen. The multimedia manager 2443 may obtainformats necessary to play various media files and use a codecappropriate for a format to perform encoding or decoding on media files.The resource manager 2444 may manage resources, such as source code ofat least one of the applications 2470, memory or storage space.

The power manager 2445 may operate together with a basic input/outputsystem (BIOS) to manage battery or power and provide power informationnecessary for operating the electronic device. The database manager 2446may generate, search, or vary a database to be used in at least one ofthe applications 2470. The package manager 2447 may manage installationor update of an application that is distributed in the form of a packagefile.

The connectivity manager 2448 may manage wireless connectivity, such asWi-Fi or Bluetooth®. The notification manager 2449 may display or notifyan event, such as an incoming message, appointment, or proximitynotification, of the user without interfering with the user. Thelocation manager 2450 may manage locational information on theelectronic device. The graphic manager 2451 may manage graphic effectsto be offered to the user and their related user interface. The securitymanager 2452 may provide various security functions necessary for systemsecurity or user authentication. According to an embodiment of thepresent disclosure, when the electronic device 2201 has telephonycapability, the middleware 2430 may further include a telephony managerfor managing voice call or video call functions of the electronicdevice.

The middleware 2430 may include a middleware module forming acombination of various functions of the above-described components, mayprovide a specified module per type of the operating system in order toprovide a differentiated function, and may dynamically omit someexisting components or add new components.

The API 2460 may be a set of API programming functions and may havedifferent configurations depending on operating systems. For example, inthe case of Android or iOS, one API set may be provided per platform,and in the case of Tizen, two or more API sets may be offered perplatform.

The applications 2470 may include one or more applications that mayprovide functions such as, such as a home 2471, a dialer 2472, a shortmessage service (SMS)/multimedia messaging service (MMS) 2473, aninstant message (IM) 2474, a browser 2475, a camera 2476, an alarm 2477,a contact 2478, a voice dial 2479, an email 2480, a calendar 2481, amedia player 2482, an album 2483, a clock 2484, a health-care (e.g.,measuring the degree of a workout or blood sugar), or provision ofenvironmental information such as air pressure, moisture, or temperatureinformation.

According to an embodiment of the present disclosure, the applications2470 may include an information exchanging application supportinginformation exchange between the electronic device 2201 and an externalelectronic device 2202 or 2204. Examples of the information exchangeapplication may include, but is not limited to, a notification relayapplication for transferring specific information to the externalelectronic device, or a device management application for managing theexternal electronic device.

For example, the notification relay application may include a functionfor relaying notification information generated from other applicationsof the electronic device (e.g., the SMS/MMS application, emailapplication, health-care application, or environmental informationapplication) to the external electronic device 2202 or 2204. Thenotification relay application may receive notification information fromthe external electronic device and may provide the received notificationinformation to the user.

The device management application may perform at least some functions ofthe external electronic device 2202 or 2204 communicating with theelectronic device, such as turning on/off the external electronic deviceor some components of the external electronic device, or control ofbrightness of the display, and the device management application mayinstall, delete, or update an application operating in the externalelectronic device or a call service or message service provided from theexternal electronic device.

According to an embodiment of the present disclosure, the applications2470 may include a health-care application of a mobile medical devicedesignated according to an attribute of the external electronic device2202 and 2204, an application received from the external electronicdevice 2202 and 2204, and a preloaded application or a third partyapplication downloadable from a server. The names of the components ofthe program module 2410 according to the illustrated embodiment may bevaried depending on the type of operating system.

According to an embodiment of the present disclosure, at least a part ofthe program module 2410 may be implemented in software, firmware,hardware, or in a combination of at least two thereof. At least a partof the programming module 2410 may be executed by a processor 2310, andmay include e.g., a module, program, routine, set of instructions, orprocess for performing one or more functions.

The term ‘module’ may refer to a unit including one of hardware,software, and firmware, or a combination of at least two thereof. Theterm ‘module’ may be interchangeably used with a unit, logic, logicalblock, component, or circuit. The module may be a minimum unit or partof an integrated component or of performing one or more functions. Themodule may be implemented mechanically or electronically. For example,the module may include at least one of application specific integratedcircuit (ASIC) chips, field programmable gate Arrays (FPGAs), orprogrammable logic arrays (PLAs) that perform some operations, whichhave already been known or will be developed in the future.

According to an embodiment of the present disclosure, at least a part ofthe device or method may be implemented as instructions stored in acomputer-readable storage medium such as a program module. Theinstructions, when executed by a processor, may enable the processor toperform a corresponding function. The computer-readable storage mediummay be the memory 2230.

The computer-readable storage medium may include a hardware device, suchas hard discs, floppy discs, and magnetic tapes (e.g., a magnetic tape),optical media such as compact disc ROMs (CD-ROMs) and digital versatilediscs (DVDs), magneto-optical media such as floptical disks, ROMs, RAMs,or flash memories. Examples of the program instructions may include notonly machine language codes but also high-level language codes which areexecutable by various computing means using an interpreter. Theaforementioned hardware devices may be configured to operate as one ormore software modules to execute embodiments of the present disclosure,and vice versa. Modules or program modules in accordance withembodiments of the present disclosure may include at least one of theaforementioned components, omit some of them, or further includeadditional components.

Operations performed by modules, programming modules or other componentsin accordance with embodiments of the present disclosure may beperformed sequentially, simultaneously, repeatedly, or heuristically.Furthermore, some of the operations may be performed in a differentorder, or omitted, or may include additional operation(s).

The embodiments disclosed herein are disclosed for description andunderstanding of the disclosed technology and do not limit the scope ofthe present disclosure. Accordingly, the scope of the present disclosureshould be interpreted as including all changes or embodiments based onthe technical spirit of the present disclosure.

While the present disclosure has been shown and described with referenceto certain embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. An electronic device, comprising: a display; asensor; and at least one processor configured to: display a stereoscopicscreen including a left-eye screen and a right-eye screen through thedisplay, move a pointer object in the stereoscopic screen, based on amovement of the electronic device sensed by the sensor, identify whetherthe pointer object is positioned at a first position at which there isno executable object for a predetermined time in the stereoscopicscreen, based on information obtained from the sensor, and in responseto identifying that the pointer object is positioned at the firstposition for the predetermined time, move at least one object includedin the stereoscopic screen into the first position at which the pointerobject is positioned.
 2. The electronic device of claim 1, wherein theat least one object includes one or more objects for controlling thestereoscopic screen.
 3. The electronic device of claim 2, wherein thestereoscopic screen includes an execution screen of a video application,and wherein the one or more objects includes one or more first objectscontrolling the execution screen of the video application.
 4. Theelectronic device of claim 3, wherein the at least one processor isfurther configured to, in response to identifying that the pointerobject is, for the predetermined time, positioned at the first positionon a time line by which the one or more first objects move, move the oneor more first objects into the first position at which the pointerobject is positioned, the one or more first objects indicating a timepoint at which a video is played.
 5. The electronic device of claim 2,wherein the stereoscopic screen includes a webpage executed through aweb browser application, and wherein the one or more objects includesone or more second objects controlling the webpage.
 6. The electronicdevice of claim 5, wherein the at least one processor is furtherconfigured to, in response to identifying that the pointer object ispositioned at the first position in the webpage for the predeterminedtime, move a third object which is positioned in line with the pointerobject in an upward, downward, right, or left direction among the one ormore second objects into the first position at which the pointer objectis positioned.
 7. The electronic device of claim 1, wherein the at leastone object includes one or more objects for controlling a function ofthe electronic device.
 8. The electronic device of claim 7, wherein theone or more objects includes one or more fourth objects for controllinga brightness level of the display.
 9. The electronic device of claim 1,wherein the at least one processor is further configured to, in responseto identifying that the pointer object is positioned at the firstposition for the predetermined time, control the display to display theat least one object in the stereoscopic screen.
 10. The electronicdevice of claim 1, wherein the at least one processor is furtherconfigured to, in response to moving the at least one object included inthe stereoscopic screen into the first position at which the pointerobject is positioned, perform a function corresponding to the at leastone object.
 11. A method for controlling an object displayed through anelectronic device, the method comprising: displaying a stereoscopicscreen including a left-eye screen and a right-eye screen through adisplay of the electronic device; moving a pointer object in thestereoscopic screen, based on a movement of the electronic device sensedby a sensor of the electronic device; identifying whether the pointerobject is positioned at a first position at which there is no executableobject for a predetermined time in the stereoscopic screen, based oninformation obtained from the sensor; and in response to identifyingthat the pointer object is positioned at the first position for thepredetermined time, moving at least one object included in thestereoscopic screen into the first position at which the pointer objectis positioned.
 12. The method of claim 11, wherein the at least oneobject includes one or more objects for controlling the stereoscopicscreen.
 13. The method of claim 12, wherein the stereoscopic screenincludes an execution screen of a video application, and wherein the oneor more objects includes one or more first objects controlling theexecution screen of the video application.
 14. The method of claim 13,wherein moving the at least one object into the first positioncomprises, in response to identifying that the pointer object is, forthe predetermined time, positioned at the first position on a time lineby which the one or more first objects move, moving the one or morefirst objects into the first position at which the pointer object ispositioned, the one or more first objects indicating a time point atwhich a video is played.
 15. The method of claim 12, wherein thestereoscopic screen includes a webpage executed through a web browserapplication, and wherein the one or more objects includes one or moresecond objects controlling the webpage.
 16. The method of claim 15,wherein moving the at least one object into the first positioncomprises, in response to identifying that the pointer object ispositioned at the first position in the webpage for the predeterminedtime, moving a third object which is positioned in line with the pointerobject in an upward, downward, right, or left direction among the one ormore second objects into the first position at which the pointer objectis positioned.
 17. The method of claim 11, wherein the at least oneobject includes one or more objects for controlling a function of theelectronic device.
 18. The method of claim 17, wherein the one or moreobjects includes one or more fourth objects for controlling a brightnesslevel of the display.
 19. The method of claim 11, further comprising, inresponse to identifying that the pointer object is positioned at thefirst position for the predetermined time, controlling the display todisplay the at least one object in the stereoscopic screen.
 20. Themethod of claim 11, further comprising, in response to moving the atleast one object included in the stereoscopic screen into the firstposition at which the pointer object is positioned, perform a functioncorresponding to the at least one object.